Summary:

The optimization_trace and other traces on BenchmarkResult are indexed by "completion event" rather than trial number. In the asynchronous case, multiple trials can complete at the same simulated time and get grouped into a single trace entry, so the trace length can be less than the number of trials. Previously there was no way to determine how many trials had completed at each trace entry.

This diff adds a `num_trials` field to `BenchmarkResult` (a `list[int]`) that records the cumulative number of completed or early-stopped trials at each completion event. In the synchronous case this is simply `[1, 2, ..., n]`. In the async case it can increase by more than 1 at a step, e.g. `[2, 4, 5, ...]`. The field is optional (`None`) for backwards compatibility with old stored results.

On `AggregatedBenchmarkResult`, the mean `num_trials` across replications is added as a new column on the `optimization_trace` and `score_trace` DataFrames (when available on all results).

Reviewed By: hvarfner

Differential Revision: D93751894

Summary:

The optimization_trace and other traces on BenchmarkResult are indexed by "completion event" rather than trial number. In the asynchronous case, multiple trials can complete at the same simulated time and get grouped into a single trace entry, so the trace length can be less than the number of trials. Previously there was no way to determine how many trials had completed at each trace entry.

This diff adds a `num_trials` field to `BenchmarkResult` (a `list[int]`) that records the cumulative number of completed or early-stopped trials at each completion event. In the synchronous case this is simply `[1, 2, ..., n]`. In the async case it can increase by more than 1 at a step, e.g. `[2, 4, 5, ...]`. The field is optional (`None`) for backwards compatibility with old stored results.

On `AggregatedBenchmarkResult`, the mean `num_trials` across replications is added as a new column on the `optimization_trace` and `score_trace` DataFrames (when available on all results).

Reviewed By: hvarfner

Differential Revision: D93751894

Summary:
Pull Request resolved: facebook#4941

The optimization_trace and other traces on BenchmarkResult are indexed by "completion event" rather than trial number. In the asynchronous case, multiple trials can complete at the same simulated time and get grouped into a single trace entry, so the trace length can be less than the number of trials. Previously there was no way to determine how many trials had completed at each trace entry.

This diff adds a `num_trials` field to `BenchmarkResult` (a `list[int]`) that records the cumulative number of completed or early-stopped trials at each completion event. In the synchronous case this is simply `[1, 2, ..., n]`. In the async case it can increase by more than 1 at a step, e.g. `[2, 4, 5, ...]`. The field is optional (`None`) for backwards compatibility with old stored results.

On `AggregatedBenchmarkResult`, the mean `num_trials` across replications is added as a new column on the `optimization_trace` and `score_trace` DataFrames (when available on all results).

Reviewed By: hvarfner

Differential Revision: D93751894

Summary:

The optimization_trace and other traces on BenchmarkResult are indexed by "completion event" rather than trial number. In the asynchronous case, multiple trials can complete at the same simulated time and get grouped into a single trace entry, so the trace length can be less than the number of trials. Previously there was no way to determine how many trials had completed at each trace entry.

This diff adds a `num_trials` field to `BenchmarkResult` (a `list[int]`) that records the cumulative number of completed or early-stopped trials at each completion event. In the synchronous case this is simply `[1, 2, ..., n]`. In the async case it can increase by more than 1 at a step, e.g. `[2, 4, 5, ...]`. The field is optional (`None`) for backwards compatibility with old stored results.

On `AggregatedBenchmarkResult`, the mean `num_trials` across replications is added as a new column on the `optimization_trace` and `score_trace` DataFrames (when available on all results).

Reviewed By: hvarfner

Differential Revision: D93751894

Summary:
Pull Request resolved: facebook#4941

The optimization_trace and other traces on BenchmarkResult are indexed by "completion event" rather than trial number. In the asynchronous case, multiple trials can complete at the same simulated time and get grouped into a single trace entry, so the trace length can be less than the number of trials. Previously there was no way to determine how many trials had completed at each trace entry.

This diff adds a `num_trials` field to `BenchmarkResult` (a `list[int]`) that records the cumulative number of completed or early-stopped trials at each completion event. In the synchronous case this is simply `[1, 2, ..., n]`. In the async case it can increase by more than 1 at a step, e.g. `[2, 4, 5, ...]`. The field is optional (`None`) for backwards compatibility with old stored results.

On `AggregatedBenchmarkResult`, the mean `num_trials` across replications is added as a new column on the `optimization_trace` and `score_trace` DataFrames (when available on all results).

Reviewed By: hvarfner

Differential Revision: D93751894

Summary:

The optimization_trace and other traces on BenchmarkResult are indexed by "completion event" rather than trial number. In the asynchronous case, multiple trials can complete at the same simulated time and get grouped into a single trace entry, so the trace length can be less than the number of trials. Previously there was no way to determine how many trials had completed at each trace entry.

This diff adds a `num_trials` field to `BenchmarkResult` (a `list[int]`) that records the cumulative number of completed or early-stopped trials at each completion event. In the synchronous case this is simply `[1, 2, ..., n]`. In the async case it can increase by more than 1 at a step, e.g. `[2, 4, 5, ...]`. The field is optional (`None`) for backwards compatibility with old stored results.

On `AggregatedBenchmarkResult`, the mean `num_trials` across replications is added as a new column on the `optimization_trace` and `score_trace` DataFrames (when available on all results).

Reviewed By: hvarfner

Differential Revision: D93751894

Summary:

The optimization_trace and other traces on BenchmarkResult are indexed by "completion event" rather than trial number. In the asynchronous case, multiple trials can complete at the same simulated time and get grouped into a single trace entry, so the trace length can be less than the number of trials. Previously there was no way to determine how many trials had completed at each trace entry.

This diff adds a `num_trials` field to `BenchmarkResult` (a `list[int]`) that records the cumulative number of completed or early-stopped trials at each completion event. In the synchronous case this is simply `[1, 2, ..., n]`. In the async case it can increase by more than 1 at a step, e.g. `[2, 4, 5, ...]`. The field is optional (`None`) for backwards compatibility with old stored results.

On `AggregatedBenchmarkResult`, the mean `num_trials` across replications is added as a new column on the `optimization_trace` and `score_trace` DataFrames (when available on all results).

Reviewed By: hvarfner

Differential Revision: D93751894

Summary:
Pull Request resolved: facebook#4941

The optimization_trace and other traces on BenchmarkResult are indexed by "completion event" rather than trial number. In the asynchronous case, multiple trials can complete at the same simulated time and get grouped into a single trace entry, so the trace length can be less than the number of trials. Previously there was no way to determine how many trials had completed at each trace entry.

This diff adds a `num_trials` field to `BenchmarkResult` (a `list[int]`) that records the cumulative number of completed or early-stopped trials at each completion event. In the synchronous case this is simply `[1, 2, ..., n]`. In the async case it can increase by more than 1 at a step, e.g. `[2, 4, 5, ...]`. The field is optional (`None`) for backwards compatibility with old stored results.

On `AggregatedBenchmarkResult`, the mean `num_trials` across replications is added as a new column on the `optimization_trace` and `score_trace` DataFrames (when available on all results).

Reviewed By: hvarfner

Differential Revision: D93751894

Summary:
Pull Request resolved: facebook#4941

The optimization_trace and other traces on BenchmarkResult are indexed by "completion event" rather than trial number. In the asynchronous case, multiple trials can complete at the same simulated time and get grouped into a single trace entry, so the trace length can be less than the number of trials. Previously there was no way to determine how many trials had completed at each trace entry.

This diff adds a `num_trials` field to `BenchmarkResult` (a `list[int]`) that records the cumulative number of completed or early-stopped trials at each completion event. In the synchronous case this is simply `[1, 2, ..., n]`. In the async case it can increase by more than 1 at a step, e.g. `[2, 4, 5, ...]`. The field is optional (`None`) for backwards compatibility with old stored results.

On `AggregatedBenchmarkResult`, the mean `num_trials` across replications is added as a new column on the `optimization_trace` and `score_trace` DataFrames (when available on all results).

Reviewed By: hvarfner

Differential Revision: D93751894